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SECTION  1  -  GENERAL 


1.1  PURPOSE  OF  THE  TECHNICAL  REPORT 


The  purpose  of  this  final  technical  report  for  the  Product 
Performance  System  (PPFS)f  Contract  Number  F33700-83-C-0038 ,  CDRL 
#AB08,  is  to  summarize  the  work  accomplished  under  Phase  II  of 
this  contract  and  to  make  recommendations  on  how  to  proceed  with 
the  contract  objectives. 

As  stated  in  the  original  Statement  of  Work  (SOW),  the 
objective  of  the  Product  Performance  Feedback  System  was  to 
create  "a  convenient,  usable,  central  repository  of  design 
related  operational,  maintenance,  and  reliability  data,  as  well 
as  other  support  and  cost  data  to  be  collected  and  used 
throughout  the  life  cycle  of  a  weapon  system".  This  central 
repository  for  data  would  contain  all  of  the  AFLC  "lessons 
learned",  which  would  help  the  design  community  develop  a 
technology  investment  strategy  to  resolve  problems  on  existing 
and  future  weapon  systems. 

This  report  describes  the  evolution  of  the  PPFS  effort  and 
the  work  accomplished  under  the  contract.  The  report  consists  of 
four  sections.  The  first  section  provides  an  overview  of  this 
document  and  other  documents  related  to  the  PPFS  program.  The 
second  section  provides  an  overview  of  the  PPFS  project  and  a 
brief  summary  of  the  work  done  during  the  tenure  of  the 
contract.  The  third  section  describes  the  results  obtained 
during  the  PPFS  development.  The  final  section  presents  the 
conclusions  reached  and  the  recommendations  to  be  made  about  the 
PPFS  effort. 
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1.2  PROJECT  REFERENCES 


The  Product  Performance  Feedback  System  (PPFS)  project  was 
conducted  for  Headquarters,  Air  Force  Logistics  Command  (HQ 
AFLC ) ,  by  Dynamics  Research  Corporation  (DRC),  Wilmington, 
Massachusetts.  Within  HQ  AFLC,  the  Command,  Control  and 
Communication  Division  (SMW)  of  the  Logistics  Management  Systems 
Center  (LMSC)  was  responsible  for  the  technical  management  of  the 
project. 


tx 


As  the  PPFS  effort  progressed,  a  distinct  connection  between 
the  PPFS  project  and  the  Weapon  System  Management  Information 
System  (WSMIS)  program  developed.  WSMIS  is  currently  being 
developed  as  the  primary  information  system  to  support  several 
major  groups  of  AFLC  Managers.  The  evolution  of  the  PPFS  project 
included  its  incorporation  into  the  WSMIS  program,  with  specific 
ties  to  the  WSMIS  Get-Well  Assessment  Module  (GWAM) .  A  list  of 
PPFS  and  WSMIS/GWAM  project  references  is  presented  below. 

Product  Performance  Feedback  System  (PPFS)  Management  Plan, 
Dynamics  Research  Corporation,  E-9540-U,  15  March  1985 


Product  Performance  Feedback  System  (PPFS)  Project  Status  Report 
and  Program  Milestones,  Dynamics  Research  Corporation, 


E-9552-U , 

12 

November  1984 

E-9609-U , 

12 

December  1984 

E-9666-U , 

12 

January  1984 

E-9712-U, 

12 

February  1985 

E-9789-U, 

12 

March  1985 

E-9861-U, 

10  April  1985 

E-9959-U, 

10 

May  1985 

E-10036-U , 

10 

June  1985 
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AFLC  WSMIS  Ogden  Working  Group  Minutes,  Dynamics  Research 
Corporation,  E-9875-U,  12  April  1985 

WSMIS  Functional  Description,  Appendix  C:  Get-Well  Assessment 
Module ,  E-9704-U,  Dynamics  Research  Corporation,  15  February 

1985 

Product  Performance  Feedback  System  (PPFS)  Test  Plan,  Dynamics 
Research  Corporation,  E-9690-U,  1  March  1985 

Product  Performance  Feedback  System  (PPFS)  Test  Analysis  Report, 
Dynamics  Research  Corporation,  E-9877-U,  29  April  1985 

Automated  AFLC  Form  74  (Release  1.0)  Program  Maintenance  Manual, 
Dynamics  Research  Corporation,  E-9879-U,  29  April  1985 

Automated  AFLC  Form  74  (Release  1.0)  Computer  Operations  Manual, 
Dynamics  Research  Corporation,  E-9881-U,  29  April  1985 

Automated  AFLC  Form  74  (Release  1,0)  Training  Material,  Dynamics 
Research  Corporation,  E-9880-U,  29  April  1985 

Automated  AFLC  Form  74  (Release  1.0)  Users  Manual,  Dynamics 
Research  Corporation,  E-9878-U,  29  April  1985 

Product  Performance  Feedback  System  (PPFS)  Final  Report,  Dynamics 
Research  Corporation,  E-9876-U,  9  May  1985 

WSMIS  Preliminary  System/Subsystem  Specification:  Get-Well 

Assessment  Module,  Dynamics  Research  Corporation,  E-10112-U, 

28  June  1985 
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A  complete  listing  of  terms  and  abbreviations  relating  to 
this  project  is  presented  below. 


ADP 

AFLC 

CDRL 

DRC 

GWAM 

IM 

LIMFAC 

LMSC 

MIS 

PPFS 

RFP 

SIMSPO 

SOW 

SPM 

VAMOSC 

WSMIS 


Automated  Data  Processing 
Air  Force  Logistics  Command 
Contract  Data  Requirements  List 
Dynamics  Research  Corporation 
Get-Well  Assessment  Module 
Item  Manager 
Limiting  Factor 

Logistics  Management  Systems  Center 
Management  Information  System 
Product  Performance  Feedback  System 
Request  for  Proposal 
Simulator  System  Program  Office 
Statement  of  Work 
System  Program  Manager 

Visibility  and  Management  of  Operation  and 
Support  Costs 

Weapon  System  Management  Information  System 


SECTION  2  -  PROGRAM  OVERVIEW  AND  SUMMARY 


2.1  PROGRAM  OVERVIEW 

The  Product  Performance  Feedback  System  (PPFS)  originated  in 
response  to  a__need  for  a  convenient,  systematic  Automated  Data 
Processing  ‘  (ADP)  ^tool  to  feed  back  to  the  design  community  the 
lessons  learned  during  the  operational  phase  of  a  weapon 
system.  The  concept  of  PPFS  was  developed  in  accordance  with  AFR 
800-13  (Air  Force  Feedback  Program)  in  order  to  perform  the 
following  functions: 

Provide  feedback  to  the  user  of  certain  design-related 
operational,  maintenance,  and  reliability  data, 
indicating  the  performance  of  a  weapon  system  in  the 
field; 

.  •  Relate  operational  performance  data  to  current  design 
data  in  order  to  check  actual  system  performance 
against  original  design  goals;  and 

•  Relate  actual  cost  to  the  level  of  performance  of  a 
given  weapon  system.  ^ -  /  , 

Originally,  the  PPFS  study  proposed  to  accomplish  these 
functions  in  two  phases.  The  first  phase  involved  the  definition 
of  a  system  that  would  function  as  a  conduit  between  AFLC  data 
systems  and  the  design  communities.  The  second  phase  involved 
the  actual  development  of  a  prototype  work  station  to  demonstrate 
the  PPFS  concept  and  stimulate  feedback  from  the  appropriate  user 
communities. 
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Two  major  objectives  were  defined  in  the  final  modification 
of  the  PPFS  contract.  The  first  objective  was  to  expand  the 
prototype  PPFS  work  station  in  order  to  address  Simulator  System 
Program  Office's  (SIMSPO)  objectives  of  improving  simulator 
availability  prior  to  the  implementation  of  the  Contractor 
Logistics  Support  (CLS)  contracts.  This  effort  included  the 
development  of  the  Automated  AFLC  Form  74  -  Critical  Item/ 
Warstopper  (D087)  Data  Sheet.  The  second  objective  was  to 
develop  a  preliminary  system/subsystem  specification  for  the 
WSMIS  Get-Well  Assessment  Module  (GWAM). 


2.2  BACKGROUND 


The  Product  Performance  Feedback  System  contract  was  awarded 
to  Dynamics  Research  Corporation  (DRC)  on  14  June  1983  on  the 
basis  of  DRC's  6  May  1983  Proposal.  The  contract  was  divided 
into  two  phases  as  defined  in  the  original  RFP/SOW.  DRC 
initiated  work  on  Phase  I  of  the  study  immediately  after 
receiving  notification  of  the  contract  award. 


The  purpose  of  the  first  phase  of  the  contract  was  to 
evaluate  the  PPFS  concept  and  determine  the  feasibility  of 
implementing  the  concept.  The  overall  concept  is  to  use  a 
structured  methodology  consisting  of  a  consolidated  data  base  and 
an  integrated  set  of  analytic  tools,  and  systematic  procedures  for 
feeding  back  information  to  the  system  designers  for  decision¬ 
making  and  planning. 


During  Phase  I,  it  became  clear  that  the  microcomputer  work 
station  version  was  a  unique  and  revolutionary  concept  that 
represented  an  information  system  rather  than  a  data  system. 
Potential  PPFS  users  would  require  a  comprehensive  prototype  to 
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test  and  evaluate  its  unique  features.  The  decision  was  made  to 
build  a  demonstration  work  station  system  on  an  IBM  PC-XT 
microcomputer  in  order  to  introduce  the  work  station  concept  to 
the  user  community. 

The  comments  resulting  from  the  demonstration  of  the 
comprehensive  prototype  were  very  favorable.  Based  on  these 
comments,  the  comprehensive  work  station  approach  became  the 
foundation  of  the  prototype  design  detailed  in  DRC’s  13  February 
1984  Program  Plan,  Product  Performance  Feedback  System  (PPFS) 
Phase  II:  Prototype  PPFS,  E-9140-U.  This  Program  Plan  described 
the  conversion  of  the  static  demonstration  to  a  fully  working 
demonstration  system. 

About  the  same  time,  the  Weapon  System  Management 
Information  System  (WSMIS)  was  being  developed.  One  of  WSMIS's 
modules  had  a  purpose  similar  to  that  defined  in  the  PPFS 
concept.  The  concurrent  development  of  WSMIS  and  the  redirection 
of  the  PPFS  effort  from  the  original  SOW  dictated  a  revised  SOW 
of  25  July  1984  in  order  to  clarify  the  contract.  DRC  responded 
with  the  Technical  Proposal  for  the  Redirection  of  the  Product 
Performance  Feedback  System  ( PPFS ) ,SG-2755F ,  17  August  1984. 

Several  major  changes  to  the  contract  were  incorporated  in 
this  new  proposal.  The  comprehensive  work  station  concept, 
developed  on  a  microcomputer,  was  formally  defined.  DRC  was 
directed  to  continue  development  of  the  prototype  on  a  Zenith 
Z-100  microcomputer  rather  than  on  the  IBM  PC-XT  microcomputer. 
The  Zenith  had  been  established  as  the  Air  Force  Standard 
microcomputer  and  would  become  available  in  increasing  numbers 
throughout  the  Air  Force. 
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The  most  significant  contract  change  concerned  the 
relationship  of  PPFS  to  the  Get-Well  Assessment  Module  (GWAM)  of 
WSMIS.  The  redirection  of  the  contract  specified  that  the  PPFS 
technology  could  be  used  to  satisfy  the  objectives  of  GWAM.  To 
support  this  theory,  the  PPFS  prototype  was  refi.ned  to 
demonstrate  the  Get-Well  process  for  the  Simulator  Program  Office 
(ASD/YWB).  The  proposal  defined  this  demonstration  as 'follows: 

•  Identify  SPM  and  IM  requirements  for  weapon  system  Get- 
Well  data  and  analysis  tools; 

•  Demonstrate  the  application  of  PPFS  to  this  Get-Well 
process;  and 

•  Provide  SPMs  and  IMs  with  a  prototype  tool  to  actually 
define  Get-Well  Plans. 

In  addition  to  preparing  the  prototype  for  the  SIMSPO,  a 
preliminary  WSMIS/GWAM  System/Subsystem  Specification  was  to  be 
prepared  under  this  contract. 
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SECTION  3 


PROGRAM  RESULTS 


The  final  PPFS  contract  modification  contained  two  specific 
objectives:  1)  to  demonstrate  to  the  SIMSPO  a  prototype,  system 
which  would  assist  in  the  effective  management  of  simulator 
availability,  and  2)  to  prepare  a  preliminary  WSMIS  GWAM 
System/Subsystem  Specification.  This  section  describes  the  work 
performed  to  meet  these  objectives  and  the  results. 


3.1  PPFS/SIMSPO  DEMONSTRATION 

The  first  phase  in  developing  an  Automated  AFLC  Form  74  was 
developing  a  working  prototype  of  PPFS  on  an  IBM  microcomputer. 
An  extensive  effort  was  made  to  transition  the  software  from  the 
IBM  to  the  AF  standard  Zenith  Z-100  microcomputer .  The  PPFS 
technology,  resident  on  the  GFE  Zenith  Z-100  microcomputer,  was 
proposed  as  a  prototype  for  the  data  processing  requirements 
for  the  Simulator  System  Program  Office  (SIMSPO)  in  order  to 
improve  simulator  availability. 

The  commonality  between  WSMIS's  and  PPFS's  objectives  became 
apparent  during  the  investigation  into  improvinq  the  SIMSPO's 
ability  to  effectively  manage  simulator  availability.  The 
requirements  were  1)  to  identify  those  items  (limiting  factors) 
which  have  the  greatest  impact  on  simulator  availability,  and  2) 
to  develop  and  monitor  Get-Well  Plans  to  prevent  these  items  from 
limiting  simulator  availability  in  the  future.  Thus,  the  PPFS 
prototype  system,  which  was  demonstrated  to  the  SIMSPO,  mirrored, 
as  much  as  possible,  the  functions  of  the  WSMIS  Readiness 
Assessment  Module  (RAM)  and  the  Get-Well  Assessment  Module 
(GWAM)  . 
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A  demonstration  of  the  SIMSPO  MIS  requirements  was  held  on 
26  March  1985  and  incorporated  both  the  identification  of  problem 
items  and  the  Get-Well  process  in  the  PPFS  work  station.  A 
significant  portion  of  this  demonstration  was  the  Automated  AFLC 
Form  74  -  Critical  Item/Warstopper  (D087)  Data  sheet,  which  used 
the  microcomputer  technology  that  had  been  developed  as  part  of 
the  PPFS  work  station.  The  Automated  Form  74  is  in  keeping  with 
GWAM's  emphasis  on  automating  certain  labor  intensive  data 
acquisition  and  analysis  tasks. 

Based  on  the  demonstration  delivered  to  the  SIMSPO,  the  user 
community,  particularly  those  IMs  responsible  for  the  preparation 
of  the  AFLC  Form  74,  requested  that  the  Automated  Form  7  4  be 
enhanced  to  operate  on  a  stand-alone  basis. 

The  Z-100  Automated  AFLC  Form  74  was  proposed  as  an  interim 
capability  prior  to  the  implementation  of  the  mainframe 
WSMIS/GWAM  system.  In  this  manner,  the  IM  community  would  be 
provided  immediately  with  an  ADP  tool  to  meet  its  interim 
needs.  The  interim  Z-100  Automated  AFLC  Form  74  reduces  the  time 
from  eight  hours  to  one  half  hour.  Additional  file,  storage  and 
printing  capabilities  are  also  provided.  The  microcomputer 
technology,  in  conjunction  with  the  widespread  availability  of 
Z-100  microcomputers  throughout  the  Air  Force  Logistics  Command 
(AFLC),  provides  the  ideal  interim  solution  to  the  IMs  needs. 

This  functional  portion  has  been  extracted  from  the  PPFS 
prototype  and  elevated  to  the  production  level  in  the  form  of  a 
stand-alone  Automated  AFLC  Form  74  System.  A  program  diskette  is 
now  available.  The  documentation  required  by  the  contract  was 
tailored  to  support  this  production  system  rather  than  to  support 
the  SIMSPO  demonstration.  It  was  more  appropriate  to  supply  a 
Users  Manual,  Program  Maintenance  Manual  and  Computer  Operations 
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Manual  to  use  with  the  production  Automated  AFLC  Form  7  4  system 
rather  than  document  a  one-time  demonstration  prototype. 


The  Automated  AFLC  Form  74  is  designed  to  address  the  issue 
of  providing  timely  information  necessary  for  management  decision 
making.  Specifically,  the  Automated  AFLC  Form  74  consists  of  a 
software  package  designed  to  run  on  a  stand-along  basis  on  a  GFE 
Zenith  Z-100  microcomputer.  The  software  generates  an  automated 
form  process  to  assist  the  IM  community  in  more  efficiently 
preparing,  storing,  updating  and  printing  pertinent  Get-Well  data 
found  on  the  AFLC  Form  74  -  Critical  Item  Data  Sheet. 

The  overall  system  functions  as  a  user-friendly,  menu-dirven 
software  package  that  allows  the  user  to  specify  tasks  relative 
to  the  preparation  of  AFLC  Form  74.  The  system  assists  the  user 
in  such  functions  as  automatically  updating  and  storing  an 
existing  database,  creating  a  new  database,  calculating 
intermediate  asset  positions,  analyzing  the  impact  of  such 
calculations,  and  printing  a  finalized  version  of  the  form. 

This  system  was  demonstrated  at  the  AFLC  Critical  Item 
Conference  and  is  currently  being  extensively  used  at  the  Air 
Logistics  Centers  (ALCs)  located  at  Ogden,  Utah  (OO-ALC), 
Sacramento,  California  (SM-ALC),  and  Warner  Robbins,  Georgia  (WR- 
ALC).  The  users  have  commented  that  this  tool  certainly  improves 
their  ability  to  manage  critical  items  and  is  an  effective 
intermediate  solution  prior  to  WSMIS/GWAM. 
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3.2  WSMIS/GWAM  PRELIMINARY  SYSTEM  SPEC 


The  second  objective  of  the  final  phase  of  PPFS  was  to 
prepare  the  first  three  sections  of  the  WSMIS/GWAM 
System/Subsystem  Specification.  This  request  qave  DRC  the 
opportunity  to  incorporate  the  lessons  learned  from  the 
development  of  the  Automated  Form  74  into  WSMIS/GWAM. 

This  objective  was  satisfied  by  submitting  the  WSMIS 
Preliminary  System/Subsystem  Specification:  Get-Well  Assessment 
Module  (E-10112-U)  on  28  June  1985.  Comments  on  this  document, 
as  yet,  have  not  been  received. 
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SECTION  4  -  RECOMMENDATIONS  AND  CONCLUSIONS 


This  section  recommends  an  approach  to  pursuing  the  original 
objective  of  the  Product  Performance  Feedback  System,  which  was 
to  feed  back  to  the  design  community  the  lessons  learned  during 
the  operational  phase  of  a  weapon  system's  life  cycle.  Stated 
more  simply,  the  design  community  needs  a  methodology/system  to 
determine  where  and  when  their  technological  base  should  be 
applied  to  existing  and  future  weapon  systems  in  order  to 
maximize  the  improvement  in  combat  capability  (performance  and 
supportability)  and  life  cycle  cost.  This  feedback  process  is 
critical  to  developing  an  optimal  technology  development 
investment  strategy  which  relates  the  "return"  (i.e.  improved 
readiness,  sustainability  cost)  with  the  investment  (i.e.  type  of 
technology  "mix",  size  of  investment). 

Fundamental  to  this  methodology  is  the  relationship  between 
the  lessons  learned  and  the  highest  level  objectives  of  combat 
capability  and  life  cycle  cost.  It  is  not  adequate  or  sufficient 
to  recommend  that  an  item  be  improved  as  a  result  of  a  400% 
increase  in  failures.  It  must  be  known  if  this  item  is  one  of 
the  factors  limiting  the  weapon  system's  readiness,  sustain¬ 
ability  or  performance  and/or  is  a  high  contributor  to  life  cycle 
cost.  The  feedback  of  lessons  learned  that  are  directly  related 
to  these  objectives  will  help  identify  which  items  most  require 
technological  improvement. 

Within  the  Air  Force  Logistics  Command  (AFLC),  the  types  of 
lessons  learned  which  must  be  fed  back  are  those  related  to  1) 
weapon  system  supportability,  including  both  readiness  and 
sustainability,  and  2)  operation  and  support  costs.  Thus,  the 
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AFLC  data  systems  which  should  be  generating  these  lessons 
learned  are  the  Weapon  System  Management  Information  System 
(WSMIS)  for  readiness  and  sustainability  and  the  Visibility  and 
Management  of  Operation  and  Support  Costs  (VAMOSC)  for  operation 
and  support  costs.  There  should  be  a  functional  requirement  for 
both  of  these  data  systems  to  accumulate  a  historical  database  of 
the  high  driving  equipment  to  serve  as  a  source  for  AFLC  lessons 
learned. 
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